On a problem of Croft on optimally nested regular polygons

We present a solution for the largest regular m-gon contained in a regular n-gon. We find that the answer depends critically on the coprimality of m and n. We show that the optimal polygons are concentric if and only if gcd(m,n) > 1. Our principal result is a complete solution for the case where m and n share a common divisor. For the case of coprime m and n, we present partial results and a conjecture for the general solution. Our findings subsume some special cases which have previously been published on this problem. Mathematics Subject Classification (2010). Primary 51M20; Secondary 52C15.